Design space exploration of associative memories using spiking neurons with respect to neuromorphic hardware implementations

Stöckel A. Design space exploration of associative memories using spiking neurons with respect to neuromorphic hardware implementations. Bielefeld: Universität Bielefeld; 2016.Artificial neural networks are well-established models for key functions of biological brains, such as low-level sensory processing and memory. In particular, networks of artificial spiking neurons emulate the time dynamics, high parallelisation and asynchronicity of their biological counterparts. Large scale hardware simulators for such networks – _neuromorphic_ computers – are developed as part of the Human Brain Project, with the ultimate goal to gain insights regarding the neural foundations of cognitive processes. In this thesis, we focus on one key cognitive function of biological brains, associative memory. We implement the well-understood Willshaw model for artificial spiking neural networks, thoroughly explore the design space for the implementation, provide fast design space exploration software and evaluate our implementation in software simulation as well as neuromorphic hardware. Thereby we provide an approach to manually or automatically infer viable parameters for an associative memory on different hardware and software platforms. The performance of the associative memory was found to vary significantly between individual neuromorphic hardware platforms and numerical simulations. The network is thus a suitable benchmark for neuromorphic systems

Harnessing Neural Dynamics as a Computational Resource

Researchers study nervous systems at levels of scale spanning several orders of magnitude, both in terms of time and space. While some parts of the brain are well understood at specific levels of description, there are few overarching theories that systematically bridge low-level mechanism and high-level function. The Neural Engineering Framework (NEF) is an attempt at providing such a theory. The NEF enables researchers to systematically map dynamical systems—corresponding to some hypothesised brain function—onto biologically constrained spiking neural networks. In this thesis, we present several extensions to the NEF that broaden both the range of neural resources that can be harnessed for spatiotemporal computation and the range of available biological constraints. Specifically, we suggest a method for harnessing the dynamics inherent in passive dendritic trees for computation, allowing us to construct single-layer spiking neural networks that, for some functions, achieve substantially lower errors than larger multi-layer networks. Furthermore, we suggest “temporal tuning” as a unifying approach to harnessing temporal resources for computation through time. This allows modellers to directly constrain networks to temporal tuning observed in nature, in ways not previously well-supported by the NEF. We then explore specific examples of neurally plausible dynamics using these techniques. In particular, we propose a new “information erasure” technique for constructing LTI systems generating temporal bases. Such LTI systems can be used to establish an optimal basis for spatiotemporal computation. We demonstrate how this captures “time cells” that have been observed throughout the brain. As well, we demonstrate the viability of our extensions by constructing an adaptive filter model of the cerebellum that successfully reproduces key features of eyeblink conditioning observed in neurobiological experiments. Outside the cognitive sciences, our work can help exploit resources available on existing neuromorphic computers, and inform future neuromorphic hardware design. In machine learning, our spatiotemporal NEF populations map cleanly onto the Legendre Memory Unit (LMU), a promising artificial neural network architecture for stream-to-stream processing that outperforms competing approaches. We find that one of our LTI systems derived through “information erasure” may serve as a computationally less expensive alternative to the LTI system commonly used in the LMU

Binary Associative Memories as a Benchmark for Spiking Neuromorphic Hardware

Stöckel A, Jenzen C, Thies M, Rückert U. Binary Associative Memories as a Benchmark for Spiking Neuromorphic Hardware. Frontiers in Computational Neuroscience. 2017;11: 71.Large-scale neuromorphic hardware platforms, specialized computer systems for energy efficient simulation of spiking neural networks, are being developed around the world, for example as part of the European Human Brain Project (HBP). Due to conceptual differences, a universal performance analysis of these systems in terms of runtime, accuracy and energy efficiency is non-trivial, yet indispensable for further hard- and software development. In this paper we describe a scalable benchmark based on a spiking neural network implementation of the binary neural associative memory. We treat neuromorphic hardware and software simulators as black-boxes and execute exactly the same network description across all devices. Experiments on the HBP platforms under varying configurations of the associative memory show that the presented method allows to test the quality of the neuron model implementation, and to explain significant deviations from the expected reference output

Demonstrating Advantages of Neuromorphic Computation: A Pilot Study

Neuromorphic devices represent an attempt to mimic aspects of the brain's architecture and dynamics with the aim of replicating its hallmark functional capabilities in terms of computational power, robust learning and energy efficiency. We employ a single-chip prototype of the BrainScaleS 2 neuromorphic system to implement a proof-of-concept demonstration of reward-modulated spike-timing-dependent plasticity in a spiking network that learns to play the Pong video game by smooth pursuit. This system combines an electronic mixed-signal substrate for emulating neuron and synapse dynamics with an embedded digital processor for on-chip learning, which in this work also serves to simulate the virtual environment and learning agent. The analog emulation of neuronal membrane dynamics enables a 1000-fold acceleration with respect to biological real-time, with the entire chip operating on a power budget of 57mW. Compared to an equivalent simulation using state-of-the-art software, the on-chip emulation is at least one order of magnitude faster and three orders of magnitude more energy-efficient. We demonstrate how on-chip learning can mitigate the effects of fixed-pattern noise, which is unavoidable in analog substrates, while making use of temporal variability for action exploration. Learning compensates imperfections of the physical substrate, as manifested in neuronal parameter variability, by adapting synaptic weights to match respective excitability of individual neurons.Comment: Added measurements with noise in NEST simulation, add notice about journal publication. Frontiers in Neuromorphic Engineering (2019

miRPathDB 2.0: a novel release of the miRNA Pathway Dictionary Database

Since the initial release of miRPathDB, tremendous progress has been made in the field of microRNA (miRNA) research. New miRNA reference databases have emerged, a vast amount of new miRNA candidates has been discovered and the number of experimentally validated target genes has increased considerably. Hence, the demand for a major upgrade of miRPathDB, including extended analysis functionality and intuitive visualizations of query results has emerged. Here, we present the novel release 2.0 of the miRNA Pathway Dictionary Database (miRPathDB) that is freely accessible at https://mpd.bioinf.uni-sb.de/. miRPathDB 2.0 comes with a ten-fold increase of pre-processed data. In total, the updated database provides putative associations between 27 452 (candidate) miRNAs, 28 352 targets and 16 833 pathways for Homo sapiens, as well as interactions of 1978 miRNAs, 24 898 targets and 6511 functional categories for Mus musculus. Additionally, we analyzed publications citing miRPathDB to identify common use-cases and further extensions. Based on this evaluation, we added new functionality for interactive visualizations and down-stream analyses of bulk queries. In summary, the updated version of miRPathDB, with its new custom-tailored features, is one of the most comprehensive and advanced resources for miRNAs and their target pathways

BALL-SNP: combining genetic and structural information to identify candidate non-synonymous single nucleotide polymorphisms

Background: High-throughput genetic testing is increasingly applied in clinics. Next-Generation Sequencing (NGS) data analysis however still remains a great challenge. The interpretation of pathogenicity of single variants or combinations of variants is crucial to provide accurate diagnostic information or guide therapies. Methods: To facilitate the interpretation of variants and the selection of candidate non-synonymous polymorphisms (nsSNPs) for further clinical studies, we developed BALL-SNP. Starting from genetic variants in variant call format (VCF) files or tabular input, our tool, first, visualizes the three-dimensional (3D) structure of the respective proteins from the Protein Data Bank (PDB) and highlights mutated residues, automatically. Second, a hierarchical bottom up clustering on the nsSNPs within the 3D structure is performed to identify nsSNPs, which are close to each other. The modular and flexible implementation allows for straightforward integration of different databases for pathogenic and benign variants, but also enables the integration of pathogenicity prediction tools. The collected background information of all variants is presented below the 3D structure in an easily interpretable table format. Results: First, we integrated different data resources into BALL-SNP, including databases containing information on genetic variants such as ClinVar or HUMSAVAR; third party tools that predict stability or pathogenicity in silico such as I-Mutant2.0; and additional information derived from the 3D structure such as a prediction of binding pockets. We then explored the applicability of BALL-SNP on the example of patients suffering from cardiomyopathies. Here, the analysis highlighted accumulation of variations in the genes JUP, VCL, and SMYD2. Conclusion: Software solutions for analyzing high-throughput genomics data are important to support diagnosis and therapy selection. Our tool BALL-SNP, which is freely available at http://www.ccb.uni-saarland.de/BALL-SNP , combines genetic information with an easily interpretable and interactive, graphical representation of amino acid changes in proteins. Thereby relevant information from databases and computational tools is presented. Beyond this, proximity to functional sites or accumulations of mutations with a potential collective effect can be discovered

The new Felsenkeller 5 MV underground accelerator

The field of nuclear astrophysics is devoted to the study of the creation of the chemical elements. By nature, it is deeply intertwined with the physics of the Sun. The nuclear reactions of the proton-proton cycle of hydrogen burning, including the 3He({\alpha},{\gamma})7Be reaction, provide the necessary nuclear energy to prevent the gravitational collapse of the Sun and give rise to the by now well-studied pp, 7Be, and 8B solar neutrinos. The not yet measured flux of 13N, 15O, and 17F neutrinos from the carbon-nitrogen-oxygen cycle is affected in rate by the 14N(p,{\gamma})15O reaction and in emission profile by the 12C(p,{\gamma})13N reaction. The nucleosynthetic output of the subsequent phase in stellar evolution, helium burning, is controlled by the 12C({\alpha},{\gamma})16O reaction. In order to properly interpret the existing and upcoming solar neutrino data, precise nuclear physics information is needed. For nuclear reactions between light, stable nuclei, the best available technique are experiments with small ion accelerators in underground, low-background settings. The pioneering work in this regard has been done by the LUNA collaboration at Gran Sasso/Italy, using a 0.4 MV accelerator. The present contribution reports on a higher-energy, 5.0 MV, underground accelerator in the Felsenkeller underground site in Dresden/Germany. Results from {\gamma}-ray, neutron, and muon background measurements in the Felsenkeller underground site in Dresden, Germany, show that the background conditions are satisfactory for nuclear astrophysics purposes. The accelerator is in the commissioning phase and will provide intense, up to 50{\mu}A, beams of 1H+, 4He+ , and 12C+ ions, enabling research on astrophysically relevant nuclear reactions with unprecedented sensitivity.Comment: Submitted to the Proceedings of the 5th International Solar Neutrino Conference, Dresden/Germany, 11-14 June 2018, to appear on World Scientific -- updated version (Figure 2 and relevant discussion updated, co-author A. Domula added

Pattern representation and recognition with accelerated analog neuromorphic systems

Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since many of these devices employ analog components, which cannot be perfectly controlled, finding ways to compensate for the resulting effects represents a key challenge. Here, we discuss three different strategies to address this problem: the addition of auxiliary network components for stabilizing activity, the utilization of inherently robust architectures and a training method for hardware-emulated networks that functions without perfect knowledge of the system's dynamics and parameters. For all three scenarios, we corroborate our theoretical considerations with experimental results on accelerated analog neuromorphic platforms.Comment: accepted at ISCAS 201